The present invention relates to the field of video signal processing. More specifically, the present invention relates to the field of enhancing the resolution of digital video images, particularly when those images are upconverted from standard definition to high definition. The present invention provides, among other things, an improved interpolation method for adding scan lines between existing lines of a video image.
Video imaging and programming are very popular and important in modem society and are widely used for a number of purposes such as education and entertainment. Because of this importance and popularity, there are many means and methods in use for distributing and recording video images and video programming. For example, video programming is broadcast to television receivers over the air and through cable television systems. Video programming is also recorded and distributed on video tapes and disks. Video programming may be transmitted as streaming data or a compressed file that moves over a computer network, such as the Internet. Static video images may also be stored and distributed in all these ways. These many different methods and means of storing and distributing video programming and images attest to their importance and popularity.
In addition to wanting a large selection of readily available video programming and images, consumers and viewers also want video programming and images that are of the highest possible quality. The images as displayed on televisions, video monitors and other display devices should be as crisp and clear as possible. Consequently, much technology is devoted to devices and methods that enhance the resolution and/or appearance of a video image as displayed on a monitor or display device.
For example, the television industry is in the process of migrating from a standard definition format to a xe2x80x9chigh definitionxe2x80x9d or xe2x80x9cHDxe2x80x9d format. HD television signals contain significantly more visual data and thus are able to provide picture quality that is far beyond what has been conventionally available to television viewers. However, receiving and using HD television signals requires a television set that is constructed to handle the additional amounts of video data provided by the HD formatted television signal. Additionally, the HD video signal, when generated, includes a larger amount of visual data, e.g., more pixels or more scan lines per image.
This poses a problem with regard to the vast amount of video programming, movies, television shows, etc. that have been produced in standard definition. Clearly the television industry and video consumers would be reluctant to move into the HD regime if doing so meant that all previous standard definition programming could no longer be viewed on their HD television sets. Consequently, there is a process for xe2x80x9cupconvertingxe2x80x9d standard definition images and programming into a form that can be displayed on an HD television system.
Before any standard definition programming can be shown on an HD television system, it must go through a process that generates additional visual data from the existing data to supply the total amount of image data that will be expected and required by the HD system. In one example, this is done by spacing apart each horizontal scan line of a standard definition video image and filling in each such spacing with an interpolated scan line, i.e., a line made by averaging pixels vertically from the lines above and below.
This process is illustrated in FIG. 1. As shown in FIG. 1, a first scan line (10) made up of a number of pixels is separated from a subsequent scan line (12). An interpolated line (11) is then created between the two. To create the data for this line, pixels from the parent lines (10 and 12) are averaged. Each vertical pixel set, e.g., (13), is averaged. Pixel (A) from the first parent line (10) and pixel (B) from the second parent line (12) are averaged to create pixel (C) in the interpolated line. This procedure is repeated along the lines until the interpolated line (11) is fully created.
Thus, the process of upconversion can be used to reformat standard definition video programming into a form that can be used by a high definition system. The principles of upconversion can also be used to decompress video images or programming that have been compressed, i.e., downsampled, for transmission. Compression, i.e., a reduction in the amount of data being stored or recorded, makes it easier to transmit or broadcast the remaining, xe2x80x9ccompressedxe2x80x9d data.
In downsampling, for example, the data required for a visual image, and consequently the resolution of the image, is reduced in the following manner. A sample area of the image containing a number of pixels is replaced by a single pixel. The color and other characteristics of the single replacement pixel may be an average of the color or other characteristics of all the pixels in the sample area. Consequently, the number of pixels in the image is reduced by a desired factor. However, resolution is also obviously lost because the amount of data describing the picture is reduced by the averaging.
Using the principles described above, Upconversion can be applied to effectively reverse this process, i.e., decompress a downsampled image or video program. However, because downsampling averages image data to generate fewer pixels, an amount of image data is lost in downsampling that cannot be recovered. Upconversion then uses further averaging to guess at what intervening pixels should look like. Consequently, an image that has been downsampled and then upconverted will not be as defined or clear a picture as the original.
Interpolation of video data for upconversion can be particularly inaccurate where there are object edges or other visual transitions with a diagonal geometry in the video image. Because the location of such an edge changes from horizontal line to horizontal line in the image, perhaps in a non-linear manner, averaging pixel data vertically to interpolate a line containing the edge will frequently blur the edge and decrease the appearance and effective resolution of the resulting image.
This effect is illustrated in FIGS. 2 and 3. FIG. 2 illustrates two succeeding scan lines (10 and 12) of a video image. An edge, line or other visual transition (20) in the image cuts diagonally across these two lines (10 and 12). FIG. 3 illustrates the creation of an interpolated line (11) between the two parent lines (10 and 12). As shown in FIG. 3, a vertical pixel averaging results in a gray area (21) being formed around the break in the edge (20). Obviously, this is approach does not preserve continuity or definition in the appearance of the edge (20).
Consequently, there is a need in the art for an improved means and method of interpolating lines in a visual image that is being upconverted, particularly where those lines contain an object edge, line or other visual transition with a diagonal geometry.
The present invention meets the above-described needs and others. Specifically, the present invention provides an improved means and method of interpolating lines in a visual image that is being upconverted, particularly where those lines contain an object edge, line or other visual transition with a diagonal geometry.
Additional advantages and novel features of the invention will be set forth in the description which follows or may be learned by those skilled in the art through reading these materials or practicing the invention. The advantages of the invention may be achieved through the means recited in the attached claims.
The present invention may be embodied and described as a video signal upconversion system that includes: a detector for detecting visual transitions in a digital video signal; a comparator, connected to the detector, for matching visual transitions in successive scan lines in the digital video signal; variable delay filters controlled by a skew signal from the comparator for delaying or accelerating the digital video signal so as to shift a visual transition in a scan line toward an average location between matched portions of the transition in successive scan lines; and a summing circuit for adding successive scan lines as skewed by the variable delay filters to generate an interpolated scan line.
Preferably, the system further includes a controlled buffer for interlacing scan lines from the video signal with interpolated scan lines output by the summing circuit. Preferably, the detector is a luma detector.
The video signal may be a color video signal. In which case, the variable delay filters shift visual transition in scan lines for each color component of the video signal according to the skew signal. In this way, a color video signal can be upconverted by the system.
The present invention also encompasses the methods of making and using a system like that described above. For example, the present invention encompasses a method of upconverting a video signal by (1) detecting visual transitions in a digital video signal; (2) matching visual transitions in successive scan lines in the digital video signal; (3) delaying or accelerating the digital video signal so as to shift a visual transition in a scan line toward an average location between matched portions of the transition in successive scan lines; and (4) adding successive scan lines as skewed by the delaying or accelerating to generate an interpolated scan line.
Finally, it should be noted that the present invention can be implemented as hardware, firmware or software. Consequently, the present invention also encompasses computer-readable instructions stored on a computer-readable medium that a computer or processor to upconvert a video signal by: detecting visual transitions in the digital video signal; matching visual transitions in successive scan lines in the digital video signal; delaying or accelerating the digital video signal so as to shift a visual transition in a scan line toward an average location between matched portions of the transition in successive scan lines; and adding successive scan lines as skewed by the delaying or accelerating to generate an interpolated scan line. The computer-readable may further cause the computer or processor to interlace scan lines from the video signal with interpolated scan lines to generate a high-definition video signal. Finally, where the input video signal is a color video signal, the computer-readable instructions may further cause the computer or processor to shift visual transitions in scan lines for each color component of the video signal.